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(57) Abstract 

A control system (1,2) for control- 
ling an image array sensor (3) and con- 
trolling communication between the image 
array sensor (3) and a microcontroller (5) 
by way of a serial communication interface 
(6). The control system (1, 2) is able to ef- 
ficiently control various aspects of the im- 
age array sensor (3) such as windowing, 
mode of operation, sensitivity as well as 
other parameters in order to reduce the data 
throughput. An important aspect of the in- 
vention relates to the fact that the control 
circuit (1,2) can be rather easily and effi- 
ciently configured in CMOS with relatively 
few output pins which enable the control 
circuit (1, 2) to be rather easily and effi- 
ciently integrated with CMOS based image 
array sensors (3) and even the microcon- 
troller (5) to reduce the part count and thus, 
the overall cost of the system. 
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CONTROL CIRCUIT FOR IMAGE ARRAY SENSORS AND AUTOMATIC HEADLAMP CONTROL 

Technical Field 

The present invention relates to a control system and more 
particularly to a control system for controlling an image array sensor having a 
5 predetermined number of pixels and controlling communication between the image 
array sensor and a microcontroller by way of a serial communication interface 
which allows various subsets of the pixels or subwindows as well as the mode of 
operation of the image array sensor to be efficiently and economically controlled 
by way of a circuit that is adapted to be formed as an application specific 
10 integrated circuit (ASIC) and even integrated with the image array sensor and the 
microcontroller to form a custom IC. 

Background Art 

Image array sensors are generally known in the art. Both photogate 
and photodiode image array sensors are known. Examples of such image array 

15 sensors are disclosed in U.S. Patent Nos. 5,386,128 and 5,471,515 and 2/SPIE 
VOL. 1900, "ACTIVE PIXEL SENSORS: ARE CCD'S DINOSAURS?" by Eric 
R. Possum, pages 2-14, July 1993, hereby incorporated by reference. Active 
pixel image array sensors are also known, for example, as manufactured by 
PHOTOBIT LLC, La Crescenta, California. Such active pixel image array sensors 

20 are normally provided with a predetermined number of pixels forming a window, 
for example a 50 X 50 window. 

There are several important control considerations related to such 
active pixel image array sensors. One important consideration relates to what is 
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commonly known as a windowing function. A windowing function relates to the 
ability to control the images of subwindows within the image array sensor for 
various purposes. For example, an active pixel image array sensor may be used 
for headlamp and tail lamp sensing as part of an automatic headlamp dimming 

5 system. More particularly, in one embodiment of the invention, the image sensor 
is divided up into virtually two identical sized independently positioned 
subwindows in one frame through different filters; one for imaging headlamps and 
the other for imaging tail lamps. In such an application, one row may be scanned 
from the first subwindow and a corresponding row from the other window. The 

10 process is repeated until all of the rows in the subwindows have been scanned. 

The windowing function may also be used to control the data 
throughput of the system. For example, in the application discussed above, it is 
necessary to discriminate noise, such as road signs and street lamps. In such an 
application, a harmonic analysis may be used to determine if an AC power Ime 

15 voltage is present. In such an application, the strongest harmonic is normally 120 
Hz for 60 Hz line voltage in the U.S. and a 100 Hz for 50 Hz line voltage in 
Europe. In order to utilize a Fourier series analysis to detect the 100 and 120 Hz 
frequency components, the data must be sampled at a rate which is generally more 
than twice either frequency and divides equally into 1/50 second and 1/60 second 

20 full cycle. For example, 6 uniformly spaced samples may be taken at a rate of 300 
samples per second for the 50 Hz line frequency and 5 samples at the same 300 
sample per second rate for a 60 Hz line frequency. The 300 sample per second rate 
is about 10 times the usual 30 sample per second frame rate often used for video 
cameras. To avoid excessively high data throughput rates, the frame size may be 

25 limited to a relatively small size, for example, as small as 2 pixels by 2 pixels. 

The windowing feature can also be used for alignment of the 
system. For example, a useful field of view for sensing oncoming headlamps of 
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an approaching vehicle is approximately 10° in elevation by 30° in width. 
However, in such a system it is preferable to allow for some error in the optical 
alignment of the sensor within the vehicle. For example, a sensor with a 13° 
elevational field of view may be provided to allow for a 3 ° misalignment and still 

5 view the proper 10° elevational range. The windowing feature allows the required 
10° field of view to be scanned reducing, for example, better than 20% of the 
image processing data throughput for the function. In particular, in order to obtain 
a proper field of view, a calibration measurement may be taken after the system 
is mounted. The field of view may also be based on the average position of the 

10 image of oncoming headlamps or on an average of the position of a portion of the 
roadway normally illuminated by the controlled vehicles own headlamps which 
enables the system to dim the controlled vehicles headlamps based upon an 
oncoming vehicle whose headlamps normally appear at an elevation, normally 
only a few degrees above the upper extent of the portion of the roadway 

15 illuminated by the controlled vehicles own headlamps. 

Another important consideration in an application utilizing an active 
pixel image array sensor is the ability to control the sensitivity of the device. For 
certain applications, it may be necessary to adjust the sensitivity of the system in 
order to avoid saturation of the image array sensor. For example, in such an 

20 application, the image of headlamps from an oncoming vehicle appear as bright 
spots in the field of view. If the sensitivity of the image array sensor is set too 
high, the particular pixels which image the headlamps will saturate so that the 
actual intensity cannot be determined. In such a situation, since the intensity of the 
headlamp image is a general indicator of the distance of an oncoming vehicle, such 

25 information is lost. Secondly, the bright image of the headlamps from an 
oncoming vehicle is a good indication to the system that the sensed image is from 
an oncoming vehicle rather than being illuminated or reflected from an object 
illuminated by the controlled vehicles own headlamps. 
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As such, there is a need to provide improved control of the window 
size as well as the modes of operation for such active image array sensors. In 
addition to sensitivity and mode of operation adjustments, other parameters, such 
as the frame read repetition timing and the number of frames to be read, also need 
5 to be controlled. Thirdly, an important consideration is the ability to make such 
changes rather rapidly. 

Another important consideration with such systems is the rather 
limited space for such control circuits. For example, the automatic headlamp 
dimming system is preferably located in the housing which shares a mounting 

10 bracket with the rearview mirror. In such an application, space is rather limited. 
Moreover, as with any control circuit, it is always preferred to reduce the number 
of components in the circuit which normally reduces the cost considerably. For 
example, the active pixel image array sensors as discussed above are based on 
CMOS technology. Accordingly, there is a need to develop circuitry which can be 

15 integrated with the image array sensor as well as the microcontroller itself. 

Summary Of The Invention 

It is an object of the present invention to solve various problems 
associated with the prior art. 

It is yet a further object of the present invention to provide a control 
20 system for an image array sensor. 

It is yet a further object of the present invention to provide a control 
system for controlling the communication between an image array sensor and a 
microcontroller. 
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It is yet a further object of the present invention to provide a control 
circuit for controlling various functions of an image array sensor, such as an active 
pixel image array sensor. 

Briefly, the present invention relates to a control system fpr 
5 controlling an image array sensor and controlling communication between the 
image array sensor and a microcontroller by way of a serial communication 
interface. The control system is able to efficiently control various aspects of the 
image array sensor, such as windowing, mode of operation, sensitivity as well as 
other parameters in order to reduce the data throughput. An important aspect of 
10 the invention relates to the fact that the control circuit can be rather easily and 
efficiently configured in CMOS with relatively few output pins which enables the 
control circuit to be rather easily and efficiently integrated with CMOS based 
image array sensors as well as a microcontroller to reduce the part count and thus, 
the overall cost of the system. 

15 Brief Description Of The Drawings 

These and other objects of the present invention will be readily 
imderstood with reference to the following Specification and attached drawings, 
wherein: 

FIGURE 1 is a top view illustrating the headlamp emission pattern 
20 of a conventional high beam headlamp. 

FIGURE 2 is a side cross sectional view of the optical system which 
forms a part of the present invention illustrating light rays incident at a vertical 
angle within the desired field of view. 
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FIGURE 3 is similar to FIG. 2 illustrating the light rays incident 
at a vertical elevation angle beyond the desired field of view. 

FIGURE 4 is a top cross sectional view of the optical system 
illustrated in FIG. 1 illustrating the light rays at a horizontal angle within the 
5 desired field of view. 

FIGURE 5 is a block diagram of the automatic head light dimming 
system in accordance with the present invention. 

FIGURE 6 is an overall flow diagram of the image processing in 
accordance with the present invention. 

10 FIGURE 7 is a flow diagram illustrating the method for detecting 

tail lamps of vehicles within the desired field of view. 

FIGURE 8 is a flow diagram for detecting headlamps from other 
vehicles within the desired field of view. 

FIGURE 9 is a block diagram illustrating an exemplary application 
15 of the control circuit in accordance with the present invention. 

FIGURES 10a- 10c are a schematic diagram of the block diagram 
illustrated in FIG. 9. 

FIGURES 11a and 1 lb are block diagrams of the registers and the 
logic used for a portion of the control circuit in accordance with the present 
20 invention. 
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FIGURES 12a and 12b are a top level diagram of the control circuit 
in accordance with the present invention. 

FIGURE 12c is a schematic diagram of the SerBlk portion of the 
control circuit in accordance with the present invention. 

5 FIGURES 12d-12g are schematic diagrams of the CanCtrl portion 

of the control circuit in accordance with the present invention. 

Best Mode For Carrying Out The Invention 

The present invention relates to a control system for controlling an 
image array sensor, such as an active pixel image array sensor, as described 

10 above, for example, a PHOTOBIT photodiode image array sensor and providing 
a relatively efficient serial communication interface with a microcontroller. 
Although the system is described and illustrated with reference to a photodiode 
image array sensor, the principles of the present invention are also applicable to 
other image array sensors, such as photogate type image array sensors, for 

15 example, as described above. The control system described below is adapted to be 
integrated with existing CMOS image array sensors and even a microcontroller. 
The serial conmnmication interface between the microcontroller and the image 
array sensor provides for a relatively low pin coimt device. For example, an 
embodiment may integrate the control logic; the programmable row, column and 

20 timing controller; the photo array sensor and associated biasing network; and the 
amplifier and analog to digital converter which represent blocks 1, 2, 3, and 4 of 
FIG. 9. Integration of these functions leaves only the microcontroller block 5 as 
a major separate block. A ground pin, a V+ supply pin, an oscillator input pin, 
and three pins for a serial interface to the microcontroller may be all of the pins 
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which are required. Even with a V- supply and a serial output pin for a diagnostic 
function, this requires only 8 pins. 

The control system in accordance with the present invention is 
illustrated in FIGS. 9-12g. An exemplary application of the control system in 

5 accordaace with the present invention is illustrated in FIGS. 1-8. However, it is 
to be understood that the principles of the invention are not limited to such an 
application. In particular, the control system in accordaace with the present 
invention is adapted to be utilized in virtually any application of an image array 
sensor in which one or more of the image array sensor parameters, such as 

10 window size, frame and the various other parameters described above need to be 
controlled. 

AUTOMATIC HEADLAMP DIMMING SYSTEM 

An automatic headlamp dimming system is adapted to provide 
automatic headlamp dimming in accordance with the Department of Transportation 

15 (DOT) regulations to provide an intensity of 40,000 cd at 3°, 32,050 cd at 6°, 
1,500 cd at 9° and 750 cd at 12°. An example of such an emission pattern is 
illustrated in FIG. L The automatic headlamp dimming system which may be 
used with the present invention includes an optical system as illustrated in FIGS. 
2-4 and an image processing system as illustrated in FIGS. 5-8. In order to enable 

20 the high beam headlamps to remain on for the longest reasonable time without the 
subjecting the driver of another vehicle to excessive glare, the automatic headlamp 
dimming system in accordance with the present invention controls the vehicle high 
beam headlamps as a function of the distance as well as the horizontal angular 
position of the other vehicle relative to the controlled vehicle. As will be 

25 discussed in more detail below, the optical system is adapted to discriminate 
between headlamps and tail lamps of other vehicles. The light rays from the 
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headlamps and tail lamps of other vehicles are spatially segregated on a pixel 
sensor array to provide increased discrimination of headlamps and tail lamps 
relative to other ambient light sources, such as road signs and reflections from 
snow and the like. The optical system enables both the horizontal and vertical 

5 position of incident lights sources to be determined within the field of view of the 
optical system. The image processing system processes the pixels to provide for 
automatic control of the headlamps as a function of the distance and horizontal 
angular position of another vehicle relative to the control vehicle. As such, the 
system in accordance with the present invention is adapted to provide optimal 

10 control of the vehicle high beam headlamps by allowing the high beam headlamps 
to remain on for as long as possible while preventing the driver of the other 
vehicle from being subjected to an undue amount of glare. 

OPTICAL SYSTEM 

Referring to FIGS. 2-4, the optical system includes a pair of lenses 
15 103 and 104, a lens holder 105 and an image array sensor 106. As best shown in 
FIGS. 2 and 3, the lenses 103 and 104 are vertically spaced apart in order to allow 
imaging of the same field of view onto different portions of the array. The lenses 
103, 104 image generally the same fields of view because the distance between the 
lenses 103, 104 is relatively small relative to the light sources within the field of 
20 view of the device. 

The lens 103 may be formed with a red filter dye for transmitting 
light with wavelengths greater than 600 nm and focusing red light rays 101 from 
tail lamps onto one half of the image array sensor 106. The red filter dye causes 
the lens 103 the absorb all light rays at the blue end of the visible spectrum and 
25 transmit light rays at the red end of the spectrum. As such, the amount of light 
transmitted from non-red light sources, such as headlamps, is greatly reduced 
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while light rays from tail lamps are fully transmitted through the lens 103. As 
such, the relative brightness of the light rays from tail lamps imaged onto the 
image array sensor 106 is greatly increased. 

The lens 104 may be formed with a cyan filtered dye for 
5 transmitting light with wavelengths less than 600 nm. The lens 104 is used to 
focus the light rays onto the other half of the image array sensor 106. The cyan 
filter dye has a complementary effect to the red filter described above. In 
particular, the cyan filter dye transmits light from the blue end of the visible 
spectrum while absorbing light from the red end of the spectmm. As such, most 
10 of the light from sources, such as head lights, are transmitted through the lens 104 
while virtually all of the light emanating from tail lamps is blocked. 

Both headlamps and tail lamps emit a substantial amount of infrared 
light. By utilizing lenses with a filter dye or separate filters which inhibit light at 
wavelengths greater about 750 nm, the infrared light transmitted by the headlamps 
15 and tail lamps will be substantially blocked by the lenses 103 and 104. By 
eliminating infrared light, the ratio between intensity between red lights imaged 
through the red filter and red light imaged through the cyan filter will be 
substantially increased. 

The use of the red and cyan dyes for the lenses 103 and 104 is 
20 merely exemplary. The filter characteristics of the lenses 103 and 104 are selected 
to optimize the sensitivity of the device to specific light sources. For example, the 
headlamps or tail lamps in new vehicles may be replaced with altemative light 
sources with different spectral composition, for example, with high intensity 
discharge headlamps and light emitting diode tail lamps requiring different filter 
25 characteristics. Depending on the spectral characteristics of the headlamps and tail 
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lamps, transparent lenses 103 and 104 may be utilized with separate color filters. 

The lenses 103 and 104 may be formed as acrylic spherical lenses. 
Alternatively, the lenses 103 and 104 may be formed as aspherical lens in order 
to minimize color dispersion and spherical aberration present with spherical lens. 
5 Complex lenses formed from both spherical and aspherical lenses are also 
contemplated. 

A smgle lens may also be used in place of the separate lenses 103 
and 104. The use of a single lens may be used to image the field of view onto a 
full or partial color image array sensor containing pigmentation on the individual 
10 pixels in the array. 

As shown best in FIGS. 2 and 3, the horizontal distance between 
the two lenses 103 and 104 and the unage array sensor 106 is slightly different. 
Offsetting of the two lenses 103 and 104 compensates for the color dispersion 
created as a result of the fact that the index of refraction of materials varies with 
15 the wavelength of light transmitted through it. Because the two lenses 103 and 
104 transmit different portions of the visible spectrum, the distance between the 
lenses 103 and 104 and the image array sensor, 106 is optunized to minimize the 
dispersion for the band of light transmitted by each of the lenses 103 and 104. 

20 As mentioned above, the light rays 101 transmitted through the lens 

103 are unaged onto one-half of the image array sensor 106 while the light rays 
102 transmitted through the lens 104 are imaged onto the other half of the unage 
array sensor 106. In order to provide such spatial segregation of the light rays 
transmitted through the lenses 103 and 104, the lens holder 105 is provided with 

25 cutouts 107 and preferably formed or coated with a light absorbing material. 
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These cutouts 107 prevent light rays beyond the desired maximum vertical angle 
transmitted through the red lens 103 from being imaged onto the portion of the 
image array sensor 106 reserved for the light rays 102. Conversely, the cutouts 
107 also prevent light rays transmitted through the lens 104 from being imaged 
5 onto the portion of the image array sensor 106 reserved for the light rays 101. 

The field of view of the optical system is defined by the 
configuration of the lenses 103 and 104 and the cutouts 107 relative to the image 
array sensor 106. For example, an exemplary field of view of 10 degrees in the 
vertical direction and 20 degrees in the horizontal directions may be created by the 

10 configuration set forth below. In particular, for such a field of view, the lenses 
103 and 104 are selected with a diameter of L5 mm with a small portion cut away 
to allow the lenses 103 104 to be positioned such that their centers are separated 
by 1 .0 mm. The lens 103 is positioned 4.15 mm from the image array sensor 106 
while the lens 104 is positioned 4.05 mm away. Both the front and rear surface 

15 radii of the lenses 103 and 104 are 4,3 millimeters with a 0.2 millimeter thickness. 

As best shown in FIGS. 3 and 4, circular cutouts 108 are formed 
in the lens holder 105. A pair of generally rectangular apermres 1 10 are formed 
in a rear wall 112. The rear apertures 110 are 1.6 millimeters in the horizontal 
direction and 0.8 millimeters in the vertical direction. As best shown, in FIG. 4, 
20 the cutouts 107 taper from the rear apertures 110 to the diameter of the front 
cutouts 108 to provide the field of view discussed above. 

The configuration described above is thus able to baffle light outside 
of the desired horizontal and vertical field of view. In particular, FIG. 3 
illustrates how the system baffles light rays incident at angles beyond the vertical 
25 field of view. FIG. 4 illustrates light rays being imaged onto the image array 
sensor 106 within the horizontal field of view. 
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The image array sensor 106 may be CMOS active pixel image 
sensor array for example, as disclosed in U.S. Patent No, 5,471,515, hereby 
incorporated by reference and available from Photobit LLC of La Crasenta, 
California. CMOS active pixel image sensors provide relatively high sensitivity 
and low power consumption as well as the ability the integrate other CMOS 
electronics on the same chip. The image array sensor 106 may be a 50 x 50 40 
yu m pixel array. The number of pixels in the image array sensor 106 is selected 
such that not all pixels fall within the area that the lenses 103 and 104 project 
onto. The extra pixels allow for simple correction for mechanical misalignment 
by offsetting the expected image location. 

The image array sensor 106 provides spatial information regarding 
light sources within the field of view. The number of pixels present in the array 
is selected to obtain sufficient spatial detail although the size of the array is not 
limited and may be selected and may even be dictated by physical and economic 
limitations. The image array sensor 106 must be sensitive to accurately detect tail 
lights at several himdred feet. Such sensitivity may be achieved by lengthening 
the amoimt of time the photosites in the array are exposed to light during a frame 
period. A frame period is selected to enable the array to capture and transfer a 
frame to the image processing system in a short enough time to allow the image 
processing system to detect another vehicle entering the field of view. A short 
time period also limits the amount of motion within a frame during the integration 
period and thus produces a relatively more accurate instantaneous image. 

The use of a pixel array also provides other benefits. For example, 
as mentioned above, the light integration time to capture a frame can be varied. 
Such a feature allows the system to provide optimal results in varying degrees in 
darkness. Another important aspect of an image array sensor is the ability to 
utilize subsets of the pixels within the array or an individual pixel. As such, as the 
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window size is decreased, the readout rates can be increased. Such a feature 
allows the system to discriminate ambient light sources, such as street lamps. In 
particular, such a feature allows the system to locate a light source within the 
frame and capture several samples of the light sources at a rate several times 
5 greater than 60 hz. In particular, if the samples exhibits 120 Hz intensity 
modulation, the light source is likely a street lamp or other light source powered 
from a 60 Hz AC power supply. If the light source is not modulated, the light 
source is likely powered by the vehicle's DC power supply. 

Another potential benefit of the image array sensor is that it allows 
10 the field of view immediately in front of the vehicle to imaged by a higher pixel 
resolution. Thus, the system may be configured such that the effective pixel 
resolution decreases as the angle of the vehicle relative to the control vehicle 
increases thus reducing the amount of processing time in those areas. Such a 
configuration reduces the sensitivity of the device to light sources from reflective 
15 stationary objects on the side of the road. 

An image array sensor could be manufactured in which the pixel 
pitch is varied as a function of the area in the field of view that the pixel images. 
For example, pixels imaging the space corresponding to horizontal angles within 
3 degrees of the center of the vehicle may be provided with a 10 jitm pixel pitch. 

20 Pixels imaging horizontal angles between 3 and 6 degrees may be provided with 
a 20 /xm pixel pitch, while those imagmg angles greater than 60 degrees may be 
provided with a 40 fim pitch. While such a configuration may not increase the 
sensing area, the ability the resolve detail increases; an important aspect in 
considering that relative size of a tail lamp at a relatively large distance. For 

25 example, a 4Vi inch diameter tail light at a distance of 200 feet subtends an angle 
of less than 0.11 degrees. If a 50 x 50 image array sensor is used to image a 20 
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degree field of view, the tail lamp would subtend approximately 5.7% of the total 
area imaged by the pixel. 

A tail lamp is relatively brighter than its ambient surroundings, 
however, the red light contributed by the tail light is diluted by the ambient light 

5 at such a distance. Such a factor is critical when comparing the amount of red 
light in a given area to the amount of non-red light in the same area. When the 
area of space compared is large relative to the light source, the percentage of red 
light is diminished. By comparison, if 10 fxm pixels are used in the center of the 
array 106 instead of 40 /xm pixels, the tail lamp would subtend 90% of the total 

10 areas, an improvement of 16 times. 

IMAGE PROCESSING SYSTEM 

The image processing is illustrated in FIGS. 5-8. The image 
processing system includes the image array sensor 106, a microprocessor 204, for 
example, a Motorola type HC08, a headlamp control unit 205 and a pair of 

15 headlamps 206. As mentioned above, an active pixel array sensor may be utilized 
for the image array sensor 106. Such an active pixel array sensor includes an 
image array 201 and an analog to digital converter (ADC) 202. A timing and 
control circuit 203 is used to control the image array 201 as well as the ADC 202 
to control the integration time, read out timmg, pixel selection, gain offset and 

20 other variables. The microprocessor 204 is used to analyze the data collected by 
the image array sensor 201 . The microprocessor 204 is in communication with the 
headlamp control unit, a conventional unit, implemented, for example, as a relay, 
which, in tum, controls the headlamps 206. The headlamp control unit 205, in 
turn, changes the voltage applied to the headlamp 206 to cause the high beam or 

25 bright lamp to be switched on or off. 
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The flow chart for the headlamp control is illustrated in FIG. 6. 
The system nms in a continuous cycle with occasional interrupts for absolute light 
measurements, adjustments of ADC parameters or other functions. 

At the beginning of each cycle, two images are acquired through the 
5 lenses 103 and 104, as in block 301. In step 302, the images from the lenses 103 
and 104 are analyzed to detect tail lamps. Another image is acquired in step 303 
through the lens 104. The image acquired through the lens 104 is acquired with 
a low enough gain to detect oncommg head lights while rejectmg lower light level 
reflections and nuisance light sources. After the images are analyzed, the system 
10 checks for very bright lights in the image indicating the sudden appearance of 
vehicle headlamps or tail lamps within the field of view, as is the case when a car 
turns in front of the controlled vehicle in step 305. If bright lights are sensed, the 
device dims the headlamps 206 immediately and bypasses the time verification as 
discussed below. The cycle is then repeated. If there were no bright lights, the 
15 system proceeds to step 307 to determine if there are any headlamps or tail lamps 
in the image. 

In order to confirm the presence or lack of a headlamp or tail lamp 
in a frame, an undim counter and a dim counter are used. These counters provide 
verification of a particular light source whether from a tail lamp or headlamp from 
20 consecutive frames before signaling the headlamp control unit 205 to dim or undim 
the headlamps 206, except as described above, when a bright light is detected. By 
providing verification, anomalies within the device or in the image will not cause 
spurious operation of the headlamps 206. 

The dim coimter is incremented each time a frame with a headlamp 
25 or tail lamp is detected until the number of required consecutive frames to take 
action is reached. The dim counter is set to 0 each time a clear frame is 
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processed. The undim counter is incremented with each clear frame and set to 0 
with each frame containing a headlamp or tail lamp. The actual number of 
consecutive frames required to dim or undim is determined by the overall speed 
of the device. The more frames used for verification, the less susceptible the 

5 system will be to noise and anomalies. However, the device must be able to react 
quickly to be effective so the number of verification frames is kept relatively low. 
Whenever a headlamp or tail lamp is detected in step 307, the undim coxmter is set 
to 0 in step 308. In step 309, the system checks whether the headlamp 206 high 
beams are on. If the high beams are off, no further action is required and the 

10 cycle is repeated as indicated by step 317. If the high beams are on, the dim 
counter is incremented in step 310. After the dim counter is incremented in step 
310, the system checks, in step 311, if the dim coimter has reached the number of 
consecutive frames required to dim the headlamps 206. If so, the system proceeds 
to step 306 and dims the headlamps 206 and resets both the dim and undim 

15 counters and repeats the cycle. Otherwise, the system repeats the cycle and 
proceeds to step to 317. 

In step 307, if there are no headlamps or tail lamps in the image, 
the dim counter is set to 0 in step 312. Subsequently, in step 313, the system 
determines whether the high beams 206 are on. If the high beams are on, the 

20 system repeats the cycle in step 317. In step 314 if the brights are not on, the 
undim counter is incremented. After the undim counter is incremented, the system 
checks in step 315 whether the undim counter has reached the number of 
consecutive clear frames required to activate the high beams 206. If so, the high 
beams are tumed on in step 316, and the cycle is repeated. If the undim counter 

25 is less than the required number for activating the bright headlamps 206, the 
system repeats the cycle in step 317. 
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The flow diagram for tail light processing is illustrated in FIG. 7. 
As will be discussed in more detail below, the primary method of identifying an 
object, such as a tail light, involves comparing the gray scale value of a pixel 
through the lens 103 to a gray scale value of the pixel representing the same space 

5 imaged through the lens 104. If the value of the pixel imaged through the lens 103 
is significantly higher than the value of the pixel imaged through the lens 104, the 
light source is determined to be red light. In addition to determinmg if the light 
is red, the system also checks the brightness of the red light before decidmg that 
the light is a tail lamp by determining if the gray scale value of the pixel is greater 

10 than a threshold value. As is known in the art, the brightness of a light source 
varies with the square of the distance of the light source from the observer. As 
such, an approximate determination of the distance of a leading vehicle can be 
made to determine the appropriate time to dim the headlamps. 

The threshold value may be computed in a variety of ways. For 
15 example, it can be a predetermined fixed number or a number that is a function 
of the current unage sensor and ADC settings. The threshold value can also be 
determined by computing a threshold as a factor of the average pixel mtensity of 
the entire image which would help eliminate variances caused by changing ambient 
light sources. In addition, the pixel value may be compared to the average of the 
20 pixels in the immediate area of the pixel of interest. This local average method 
prevents relatively large, moderately bright spots in the image from being seen as 
vehicle light sources. More particularly, distant tail lamps subtend less than one 
pixel and thus will only have moderate brightness. Large spots in the image with 
moderate brightness are most likely caused by reflections from large objects. 
25 Close tail lamps which subtend many pixels will have a saturated center which will 
be brighter than the surroundmg pixels allowing the same method to detect them 
as well. 
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The threshold may also be determined by varying the threshold 
spatially by way of a look up table or computation. However, the threshold 
should be determined so that dimming occurs appropriately for the dimmest tail 
lights allowed by the DOT standards. Distant vehicles are subjected to the most 

5 intense portion of the controlled vehicle high beam, thus requiring dimming only 
directly in front of the controlled vehicle as indicated in FIG. 1 . Thus, a relatively 
low threshold may be selected for light sources imaged directly in front of the 
control vehicle while a higher threshold for light sources that are not directly in 
front of the control vehicle. For example, as discussed in connection with FIG. 

10 1 , the threshold for pixels imaging the field of view 3 degrees right and left of the 
center should correspond to a light level incident on the image array sensor 106 
about 4 times as bright as the threshold for red light directly in front of the vehicle 
and 12 times as bright for vehicles at 6 degrees. Such a spatially varying threshold 
helps eliminate false tail lamp detection caused by red reflectors by making the 

15 system less sensitive to areas of to the sides of the control vehicle. 

A similar approach can be taken for varying the threshold for pixels 
in imaging areas of space and angles above and below the center. However, a 
more conservative approach can be taken when determining the tail light sensitivity 
20 relative to the vertical angle since vehicles tend to move more frequently and 
rapidly in vertical directions due to hills and bumps in the road. Therefore, 
specifying relatively tight vertical thresholds may cause the bright headlamps 206 
to switch on and off as the vehicle moves several degrees up and down. 

A hysteresis multiplier may be applied to the threshold to prevent 
25 oscillations of the headlamps 206 when the light source has a gray scale value at 
or near the threshold. Thus, if the bright headlamps 206 are off, the threshold will 
be lower for all pixels to prevent the bright headlamps from coming back on even 
if the faintest tail lamps are present in the image. However, if the bright 
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headlamps 206 are on, the threshold should be higher so that only tail lamps of 
sufficient brightness are sensed to indicate that the car is within the dimming range 
to cause the headlamps 206 to dim. 

One of the biggest problems facing the detection of the tail lamps 
is the nuisance red light reflected from comer cube reflectors commonly foimd as 
markers on the side of the road and on mail boxes. The variable threshold method 
mentioned above helps eliminate some of this noise. However, when a vehicle 
approaches a reflector at the proper angles, it is relatively impossible to distinguish 
a red reflector from a tail lamp. Fortunately, by examining successive frames and 
investigating the motion of these objects over time, such reflections can be 
filtered. By storing the location of the tail lamps and images over time or by 
sensing small region of interest where the tail lamp is located, several consecutive 
times, the device can look for rightward motion and determine if the light source 
is a reflector. Additionally, the speed at which the control vehicle overtakes a 
stationary object is much greater than the relative rate a vehicle would overtake 
another moving vehicle. Thus, the rate of increase in brightness of the object 
would be typically much greater for a stationary reflector than for another vehicle. 
This rate of change in brightness coupled with rightward horizontal motion can be 
used as signatures to reduce the number of false tail lamps detected. 

A computationally simpler method of analyzing spatial motion of 
a light source is to simply take several consecutive regions of the local region of 
interest where the light source is located. Motion in the vertical and horizontal 
directions is relatively slow for tail lamps of a leading vehicle. Sampling a pixel 
a few consecutive times to see if the tail lamp is present in all samples can 
adequately eliminate objects which rapidly move within the image. 
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The flow diagram for tail lamp processing is illustrated in FIG. 7. 
Initially, in step 318, the system ascertains if the pixel is within the tail lamp 
window. In particular, as mentioned above, red lights are imaged onto half of the 
image array sensor 106. Thus, if the pixel is not within the appropriate half of the 

5 image array sensor 106, the system proceeds to step 319 and moves to another 
pixel. As mentioned above, there are two criteria for ascertaining whether the 
image is a tail lamp. The first criteria relates to comparing the gray scale value 
of the pixel image through the lens 103 with a corresponding gray scale value for 
the same area in space imaged through the lens 104. If the gray scale value of the 

10 pixel imaged through the lens 103 is significantly larger than the gray scale value 
of the corresponding pixel imaged through the lens 104, one of the criteria for 
detecting a tail lamp is met. Thus, if the pixel of interest is within the lamp 
window as ascertained in step 318, the gray scale value of the pixel imaged 
through the lens 103 is compared with the gray scale value of a corresponding 

15 pixel imaged through the lens 104 in step 320. If the gray scale value of the pixel 
image through the lens 103 is not n% greater than the corresponding pixel imaged 
by the lens 104, the system proceeds to step 319 and examines another pixel. 
Otherwise, the system proceeds to step 321 and calculates the threshold for the 
particular pixel based on the region of space it images. For example, as discussed 

20 above, the pixel thresholds may be varied based on their spatial relationship within 
the image array sensor. 

As discussed above, the other criteria for tail lamp detection relates 
to the brightness of the pixel relative to the neighbor pixels. Thus, in step 322, 
25 the system calculates the average gray scale value of neighboring pixels. If it is 
determined in step 323 that the pixel gray scale value for the pixel imaged through 
the lens 103 is n% greater than the average gray scale value of the neighboring 
pixels, the system proceeds to step 324 and adds the pixel to the tail lamp list for 
future frames of reference. Otherwise, the system moves to step 319 and moves 
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the next pixel. In steps 325 and 326, the systems determines whether or not the 
red light detected is a tail lamp or a reflector, as discussed above. If it is 
determined that the light is a reflector, the system proceeds to step 327 and moves 
on to the next pixel. Otherwise, the headlamps are dimmed in step 328. 

5 The flow diagram for head light processing is illustrated in FIG. 8. 

Headlamp detection is similar to tail lamp detection. The primary difference is 
that only the image seen through the lens 104 is utilized. As mentioned above, the 
pixel integration time is shorter and the ADC parameters are such that the image 
only shows very bright objects, such as headlamps. Most reflections have low 

10 intensity light sources that fall well below the zero threshold of the ADC. As 
such, pixels are compared to the local average intensity of the neighboring pixels. 
Spatial variances in the thresholds may be set so that pixels corresponding to the 
center of the field of view are more sensitive than pixels on the left of the image 
(left hand drive countries). These thresholds, however, should not vary spatially 

15 to the same degree as the threshold for the tail lamps because of the relatively wide 
variance in the emission patterns observed from headlamps. In addition, due to 
the relatively higher potential for more glare to the driver of an oncoming car, the 
headlamps may be controlled and dimmed relatively more rapidly than in the case 
when a tail lamp from a vehicle traveling in the same direction is detected. 

20 Similar to the tail lamp processing circuit hysteresis may be added to prevent 
cycling of the headlamps. 

An additional concern with headlamp detection arises from the rapid 
decrease in distance between oncoming vehicles which becomes especially critical 
when an oncommg vehicle suddenly enters the controlled vehicle's field of view, 
25 for example, when tuming a comer or in a similar situation. For this reason, an 
additional flag is used to cause the vehicle to immediately dim the bright 
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headlamps and bypass any verification if the light source is above certain absolute 
high level brightness threshold. 

The primary nuisance light source complicating headlamp detection 
comes from overhead lights, such as street lights and electrically illuminated street 

5 signs. One method of eliminating such nuisance light sources is to analyze their 
motion. In particular, all overhead street lamps will move vertically upwards in 
the image as the controlled vehicle is moving. Analyzing this motion provides an 
efficient method of detecting some street lamps. Unfortunately, distant street 
lamps appear to be at almost the same elevational angles as distant head lights and 

10 the rate of vertical climb in the image does not become great until the street lamp 
is closer. However, as discussed above, street lighting is AC controlled and thus 
is subject to 120 Hz intensity modulation. Headlamps powered by DC source do 
not exhibit this characteristic. Thus, the image array sensor 106 is able to utilize 
a small number of pixels for taking several rapid consecutive readings in a 

15 window. If the window is small enough, the window can read several hundred 
frames per second. Once the light source is identified in the image, several fi-ames 
are read out at a rate of 240 Hz or higher. These readings are then analyzed to 
detect the intensity modulation. If modulation is present, the light source 
originates from an AC source and can be ignored. Alternatively, a photodiode can 

20 used in conjunction with a low pass filter to determine the ratio of light in the 
image that was AC modulated to the unmodulated light. If a significant portion 
of the light source is AC modulated, the light source present in the image is 
assumed to be from AC light. Otherwise, the light source is assumed to be fi-om 
a DC source, 

25 The flow diagram for headlamp processing is illustrated in FIG. 8. 

Initially, tiie system determines in step 329 whether the pixel is in the headlamp 
window (i.e. that portion the image array sensor 106 reserved for light arrays 
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imaged through the lens 104). If not, the system proceeds to step 330 and 
examines the next pixel. Otherwise, the system examines the pixel in step 331 to 
determine if the pixel is modulated at 120 Hz as discussed above. If so, the light 
source is assumed to be a street lamp and thus, the system proceeds to the next 

5 pixel in step 330. If the pixel is not subject to 120 Hz intensity modulation, the 
system then computes the average gray scale of neighboring pixels in step 332. 
In step 333, the system determines the threshold for the particular pixel based on 
the area of the space it images. The system next compares the gray scale value of 
the pixel with an absolute high level threshold in step 334, for example, to 

10 determine if any oncoming cars suddenly come into the field of view of the 
controlled vehicle. If so, the system proceeds to step 335 and sets a flag to cause 
immediate dimming. Otherwise, the system proceeds to step 336 and determines 
if the gray scale value of the pixel is n% greater than the average of neighboring 
pixels. If not, the system retums to step 330 and examines the next pixel. 

15 Otherwise, the system proceeds to step 337 and adds the pixel to the headlamp list 
for future frames to reference. 

The system examines light sources as discussed above in steps 338 
and 339 to determine if the light source is a street lamp. If the system determines 
that the light source is not a street lamp, the system proceeds to step 340 and sets 

20 a flag to cause dimming of the headlamps 206. If the system determines that the 
light source is a street lamp, the system proceeds to step 341 and moves on to the 
next pixel. Traditional vehicle lamp systems have the option of the bright lamps 
being either on or off. The present invention is readily adaptable for use with a 
headlamp system where the brights can be activated to a varying brightness based 

25 on the distance of other vehicles in the field of view. In such an embodiment, the 
brightness of the headlamps may be varied by various techniques including 
modulating the duty cycle of the headlamp in order to reduce or increase the 
overall brightness level. 
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Variable intensity headlamps also result in better noise filtration. 
In particular, whenever a light source is detected which causes the brightness of 
the controlled headlamps of the vehicle to be decreased other images can be 
detected to determine if the intensity of these other light sources decreases by a 

5 similar amount. If so, the system would be able to determine that the light source 
is a reflection from the vehicle's headlamps. Such information can be used as 
feedback to provide a relatively efficient means for eliminating nuisance light 
caused by reflections of the control vehicle headlamps. In such an embodiment, 
the brightness threshold discussed above would not be used. More particularly, 

10 the brightness of the brightest headlamp and tail lamp in the images is used to 
determine the brightness of the controlled vehicle's headlamps. The brighter the 
headlamps or tail lamp in the images, the dimmer the controlled headlamps. 

IMAGE ARRAY SENSOR CONTROL SYSTEM 

The control system in accordance with the present invention is 
15 adapted to economically perform the flexible windowing function for the 
exemplary application discussed above. There are three important considerations 
in the exemplary application discussed above, for example. First, there is great 
utility in being able to control window size and orientation as well as the modes 
of operation of the image array sensor; for example, in the application discussed 
20 above, the ability to switch between dual and single window modes of operation. 
Second, other parameters, such as the sensitivity, the frame read repetition timing, 
and the nmnber of frames to read all need to be controlled. Third, in the 
exemplary application discussed above, practically every new reading requires a 
major change in setting from the previous reading so it must be possible to rapidly 
25 change the settings. Even though in such an application, there is a need for 
frequent and numerous changes in operational settings, little overhead is added by 
requiring all of the settings to be written before each reading. This also eliminates 
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the need for time consuming address transmissions and for the extra complexity 
of a bus structure and address decoding in the image array control circuit. 

There are many other considerations for a control system for an 
application as discussed above. First, in order to minimize cost and to provide a 

5 compact control circuit, it is desirable to be able to integrate several if not all of 
the major components of the circuit into a single integrated circuit. In accordance 
with one aspect of the present invention, the control circuit, which, as discussed 
below, includes a serial interface to a microcontroller which can be integrated on 
the same integrated circuit as the image array sensor and can also be integrated 

10 with various other components, such as an analog to digital converter for digitizing 
the pixel readings. Ultimately, even the microcontroller may be integrated on the 
same integrated circuit chip. 

Another consideration for such a control circuit is that the optical 
15 window required for the imaging array sensor increases the package cost and 
electrical connections are also more expensive to provide in the package with the 
optical window than with conventional integrated circuit packages. For these 
reasons, it is highly desirable to use a low pin count serial interface between the 
imaging chip and the microcontroller. It is of further advantage to use the same 
20 bidirectional serial interface to issue instructions to the image sensor controller and 
to read the digitized image information which is returned. It is of further 
advantage to maintain a common serial path, serial registers and timing for 
interfacing with the serial port to the microcontroller, continuing to utilize the 
conunon serial architecture and registers for queuing the instructions, and finally 
25 for processing the instruction data to perform the required imaging operation. It 
is further advantageous to couple the steps in the successive approximation of the 
pixel reading to the same clock that is used for the above serial instruction 
processing and to synchronously transmit the successive bits of the conversion 
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back to the microcontroller, this bit-wise transmission commencing while the 
successive approximation steps for conversion of that same pixel are still in 
progress. Since a production imaging controller will need custom circuits and 
design to apply state of the art technology, the cost of the silicon area will also be 

5 relatively high at least initially. Thus, it is important for the logic which couples 
the imaging array to the microcontroller and which controls the image sensing 
process to be implemented with a modest silicon area. To meet these and other 
design objectives, a novel serial architecture is utilized to interface directly with 
a microcontroller to efficiently register the frequently changed instructions and to 

10 perform the processmg to control the rather complicated windowed readout 
function. 

In an exemplary application, 9 bytes (72 bits) of instmction data are 
shifted into 9 byte long shift register segments which are configured as one serially 
contiguous register during the serial load operation. Several eight bit long 

15 registers, each of which must be initialized to a respective value equal to or 
determined by one of the values in the main chain, are configured as branches off 
of the 72 bit chain. In one configuration of the invention, the byte wise serial 
order is changed from most significant bit first to least significant bit first in the 
first of the eight bit shift register segments. This is done because the 

20 microcontroller used transmits and receives the most significant bit first and the 
successive approximation analog to digital converter converts and transmits the 
most significant bit first. The serial incrementing circuits must, on the other hand, 
have the least significant bit transmitted first. In two other eight bit shift register 
segments, the value is pre-mcremented as it is shifted m. Thus, in the more 

25 general case the simple shift register fiinction is augmented with an additional 
processing step so that the specific register segment is loaded with a processed 
derivative of the serial input value. In an embodiment of the invention, the tuning 
for the analog to digital conversion process is common with the timing for the 



wo 99/14943 



PCT/US98/19075 



-28- 

serial, processing. A capacitor charge balance analog to digital converter may be 
sjnichronized to and in fact operated by the same clock which is used for the serial 
processing in the controller and the bits are transmitted over the common serial 

interface to the microcontrolier as each successive approximation step of the 
5 conversion is made. This saves any extra synchronization, control, and buffering 
logic that may be required when the two functions are not synchronized to operate 
together. 

The data which is serially loaded as described above preferably 
includes the following information. The numbers of the first and the last colunms 

10 of the first frame window are included. Bits indicating the separate options to 
inhibit reset of rows at the beginning of the integration period, to inhibit readout 
of the frames at the end of the integration period, and to select and process only 
the second frame are included. The last of these options is used when the dual 
window per frame feature as discussed above for processing headlamps and tail 

15 lamp unages is not required. The numbers of the first and last rows of the first 
frame window are included. The row position indicative of the cyclic row offset 
between the row of the frame window which is reset at the beginning of the 
integration period and the row of the frame window which is read are also 
included in the serial data. The row and column offsets of the second frame 

20 relative to the first are also included for use when the second frame is selected. 
Provision is made to further increase ttie integration period beyond that provided 
by the cyclic row offset by inserting an mtegral number of zero or more additional 
integration frame periods (0 to 255, for example) between the reset of a particular 
row in the frame or pair of frames and the reading of the particular row. The data 

25 also includes the number of times to read the frame or frame pairs to complete the 
uistruction. 
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The features to selectively inhibit the reset at the beginning of the 
integration period or the readout are included so that reset before integration and 
readout after integration can be handled as separate operations • The block size 
required for readout of the whole image at one time may exceed the 
5 microcontroller memory capacity. The readout operation resets only the rows 
which are read. For low light levels, it is necessary to use relatively long 
integration periods. The limited microcontroller memory may require the image 
to be broken into blocks so that the processing of each does not require more 
memory than is available. Without separate reset and read instructions, this would 
10 require very elaborate control or make it necessary to wait for the entire 
integration period to process each block. Efficient use of the sensor in low light 
conditions calls for collecting light over as much of the sensor area as possible as 
much of the time as is possible. Since for long integration periods, the timing is 
not ordinarily as critical, the integration can be placed under the control of the 
15 microcontroller if the available control modes permit. This capability is provided 
for this reason. Furthermore, since windows of non-overlapping rows do not reset 
rows outside of their range during either reset at the beginning of the mtegration 
period or reset as part of the readout function, the field may be broken into 
windows with non-overlapping rows which are reset on a staggered schedule using 
20 separate instructions which inhibit the readout function but enable the reset 
function. The microcontroller may then initiate successive readout and processing 
of these blocks each after an acceptable integration period by inhibiting the reset 
function and enabling the readout function. Either the dual or single frame modes 
may be selected. In this way, the integration period for processing of multiple 
25 blocks may be overlapped with a minimal increase in the complexity of the 
controller circuit. In the above sequence, the integration period for each instmction 
is normally set to a minimmn value since the actual integration period is 
determined by the microcontroller and a single readout cycle of the single or dual 
frames is normally selected. 
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The embodiment illustrated is configured to interface with and 
control an array of photodiode pixel sensors, for example as discussed above. In 
general, the charge at each of the photo sites in a row is drained off when the row 
is reset. Charge is then accumulated when electrons are freed as photons strike the 
photo site so that, for a relatively wide range of light levels and integrating times, 
the accumulated charge is approximately proportional to the product of the 
integrating time and the light level. Thus, the integration period as described above 
is analogous to the exposure time with a camera and fihn. 

In one embodiment, a successive approximation analog to digital 
converter is utilized and, to minimize the buffer memory and control logic, the 
bits are shifted back to the microcontroller as the successive approximations are 
made with minimal buffering provided to optimize clock timing on the serial bus 
to the microcontroller. 

Following the receipt of the read instruction data, the control system 
responds preferably on the same serial interface with the serially transmitted 
digitized readings which indicate the light level received by each of the pixels in 
each of the frames specified by the instruction. This reading sequence is repeated 
the number of times specified in the instruction. The microcontroller, for 
example, a Motorola 68HC708XL36, utilizes a direct memory access function 
which may be utilized to increase rates at which data can be transferred to the 
microcontroller from the photo sensor and to free the microcontroller for other 
functions. One drawback is that the direct memory access function of the particular 
microcontroller is limited to blocks of 256 bytes at a time. Each pixel reading is 
encoded as an 8 bit value so the 256 bytes can store readings from only 256 pixels. 
With a nominal 64 by 64 pixel array for the dual frame sensor for the rain or 
headlamp dmmier sensors, if for example 60 bits corresponds to 30 degrees, then 
a 10 degree by 30 degree field will contain 1200 pixels, and the corresponding two 
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color dual frame will contain 2400 pixels. This field may have to be read and 
processed in blocks to prevent overrunning the available memory in the 
microcontroller and before the memory limit is reached. To use the direct memory 
access feature of the selected microcontroller, the data will need to be collected in 

5 blocks of 256 bytes or less. In one embodiment, there is an interval equal to the 
time required to transmit 5 successive pixels in a row at the begiiming of each new 
row. To utilize the direct memory access feature when more than one block of data 
must be collected for a particular read instruction, the block length is preferably 
set so that blocks end on row boundaries and an interrupt mode is selected so that 

10 the microcontroller is interrupted at the end of each block which is received. The 
microcontroller then has the five pixel interval to set the direct memory controller 
to receive the next block of data. This between row delay time is used by the 
control circuit to issue the reset to the row to start its integration period and to 
transfer the charge from pixels in the row to capacitors for readout. This process 

15 is detailed in the circuit description which follows. The five pixel delay period 
between rows may optionally be made variable or changed to some other fixed 
delay period. However, the between row delay period for a given read operation 
is preferably made a consistent length even though not every function performed 
during this period is performed every time. The benefit of the consistent time 

20 period is that the control logic is less complicated as is the computation of the 
frame reading interval. Also, a variable period, depending on the pattern of the 
variation, could make repeated frame readings non-periodic which would create 
problems in the harmonic analysis. The microcontroller used may optionally be 
replaced by a digital signal processor or other programmable data processing 

25 device. 

Referring to FIG. 9, a block diagram of an exemplary control 
system which incorporates an image array sensor and microcontroller for use with 
the present invention is illustrated. The system includes a programmable 
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row-column and timing controller 1, control logic 2, an image array sensor 3, an 
amplifier and analog to digital convertor (A/D) circuit 4 and a microcontroller 5. 
The present invention relates to the control logic 2 and programmable row-colunm 
and timing control circuit 1 for controlling the image array sensor 3 and providing 

5 a bidirectional serial communication interface 6 with the microcontroller 5 . As will 
be appreciated by those of ordinary skill in the art the programmable row-column 
and timing control circuit 1 as well as the control logic circuit 2 which form the 
present invention may be implemented as an ASIC and integrated with one or 
more of the following portions of the overall circuit: microcontroller 5; image 

10 array sensor 3 and amplifier and A/D convertor circuit 4. 

FIGS. 10a- 10c are a schematic diagram of the block diagram 
illustrated in FIG. 9. As shown in FIGS. 10a- 10c, the control circuit in accordance 
with the present invention, which, as mentioned above incorporates the control 
logic 2 and programmable row-column and timing controller 1, is shown 
15 hnplemented, for example, as an ASIC XC4003E. 

FIGS. 12a- 12b is the top level schematic diagram used to generate 
the program for the ASIC XC4003E as shown in FIG. 10b. FIGS. 12a-12b along 
with the referenced lower level diagrams are implemented using Workview 
Office™, Version 7.31, Feb. 6, 1997, Viewlogic Systems, Inc. in combination 

20 with the Unified Library from Xilinx, Inc. These schematics are used to generate 
a net list which may be processed by the Xilinx Design Manager version ML3.7. 
The bit stream generated by the component programs of the Xilinx Design 
manager may be used to program the ASIC XC4003E. The Atmel AT17C65 serial 
memory in FIG. 10b is configured to store and automatically download the 

25 program to the XC4003E each time that power is first applied to the circuit. 
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FIGS .11a and 1 lb depict the registers and the data paths where the 
image array sensor instruction data is stored and manipulated for the control of the 
image array sensor. The overall function of the registers is to generate the 
sequence of row select and column select addresses required for the dual 

5 windowed readout of the sensor and to return status signals which expedite the 
control of the readout sequence. Blocks 300 and 320 in combination generate the 
column select address CSOO through CS06. Blocks 330 and 340 in combination 
generate the row select address RSOO through RS06. Block 360 counts the 
number of frames which are added to lengthen the sensor integrating time. Finally 

10 block 370 counts the number of times to read the frame or frame pair to complete 
the instmction sequence. 

There are two general modes of operation for the control circuit; 
the first is the instruction serial load mode in which the register is serially loaded 
with 72 bits (nine serially transmitted bytes) of data. The second mode of 
operation is the instmction execution mode in which the registers are reconfigured 
and selectively enabled for 8 clock cycle sequences to perform the incrementing, 
loading, and compare functions required to generate the complicated sequence of 
addresses and the status indicating outputs required to sequence the controlled 
windowed readout of the array sensor. The circuit generates binary row and 
column addresses which are decoded by the array sensor to select the appropriate 
rows and pixels at the intersection of the selected row and coliunn in order to 
perform the required control and readout functions. 

The control signals input to the blocks 300, 320, 330, 340, 360, 
and 370 are enumerated in the respective lists, 301, 321, 331, 341, 361, and 371 
25 and the signals which are output from these blocks (excluding the interconnections 

which are shown) are enumerated in the respective lists 302, 328, 332, 342, 362, 
and 372. Each of the blocks receives a clock signal CLK, a serial load signal SLD, 



15 



20 
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and a signal LAST indicating that this is the last of a group of 8 clock signals for 
which a particular block or group of blocks is enabled is used by every block 
except block 330. Each of the 72 rectangular boxes in the groups SCOxF, LCxF, 
HCxF, SROxF, LRxF, ARxF, HRxF, IFDxF, and RFCxF depict individual 

5 clocked D type flip-flops which have a clock enable. These flip-flops are all 
enabled and configured generally as one long shift register to load the 72 bit (9 
serially transmitted bytes) instruction word. Except for the 8 bit group SCOxF, the 
remaining eight flip-flop groups are sunple 8 bit shift registers each with a clock 
enable. During the load sequence, SCOxF receives bit 7 through bit 1 of each byte 

10 shifting them serially into SC07F through SCOIF so that just prior to the last of 
the eight clock pulses which shift in the byte, bit 7 is in SCOIF and bit 1 is in 
SC07F. Then on the last of the 8 clock pulses, LAST is asserted directing bit 0 
coming in last on SRCV directly into SCOOF and exchanging SCOIF with 
SC07F, SC02F with SC06F, SC03F with SC05F and leaving SC04F 

15 unchanged. This reverses the order of the bits of the byte so that successive bytes 
are serially transmitted least significant bit first from SCOOF to LCIN and the D 
mput of LC7F. This is necessary for the serial add functions performed m blocks 
320, 340, 360, and 370. The SCOxF group was placed first because its value 
remains static like SROxF during the execution phase and is not enabled during 

20 that phase. This eliminates the need to further complicate the data selectors which 
are used to reverse the transmission order from most significant bit first to least 
significant bit first. The four low order bits SCOO to SC04 are sign extended by 
repeating SC04 for the higher order bits of adder 307. This number transmitted on 
bus 303 is the column offset of the second frame relative to the first in signed, 

25 twos complement form which is added in adder 307 to the column address for the 
first frame which is generated in ACO through AC7 and transmitted on bus 304. 
The adder output is transmitted on bus 305. The signal SSF is asserted to select 
the bus 305 to which the second frame offset has been added and gate it to bus 
306. When SSF is not asserted, the signal 304 is gated directly to 306 without 
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adding the column offset value. CSOxF is a D type register into which the row 
address from 306 is clocked by the rising edge of CCLK. The number of bits in 
the CSOxF register may be changed to accommodate the niunber of columns of 
pixels in the sensor. 32 to 64 columns and rows are anticipated to be the number 

5 required for the headlamp dimmer control and the moisture sensing applications 
discussed above. The nmnber of bits in the row and column select addresses may 
be reduced for smaller numbers of rows and coliunns or increased to eight to 
accommodate up to 256 rows and or columns with few if any other changes. More 
rows or colmnns are within the scope of the invention but will require other 

10 adjustments in the word sizes. It is not necessary to stay with a word size of eight 
but this is convenient to match the byte size of the words in the microcontroller 
memory and in the 8 bit word size of the serially transmitted data. 

During the serial load stage, data selector 322 routes the output 
signal SCOO to LCIN which is the D input to the LC7F flip-flop. Data selector 

15 324 routes the identical bit stream to the D input of AC7F and data selector 325 
routes the output of LCOF to the D input of HC7F. At the completion of the 
instruction load phase, the eight bit registers LCxF and ACxF both contain the 
address of the starting or low column of the first frame and HCxF contains the 
address of the last or high column of the first frame. The count in ACxF thus 

20 begins at the lowest column address and is incremented until it matches the high 
column address. On the next pixel count after the high column address is matched, 
a copy of the low column address is again shifted into ACxF so that the 
uicremental count from the low to the high column address is repeated. Thus the 
increment pixel count is a simple serial add one operation imtil the cotmt equals 

25 the high column address and it is then a copy operation of the low column address 
into the ACxF count register for the next increment operation. 
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The logic depicted in block 320 is self contained in that it performs 
the incrementing, the comparing and the copying as required to produce the 
sequence described above. It does so in response to a series of eight cycles of the 
CLK input for which the enable pixel count EPXC input is high. On the first of 

5 the eight CLK cycles, FIRST is asserted and is used by the serial add one circuit 
326 to force the addition of one to the least significant (first) bit of the ACxF 
count. The serial add one circuit 326 contains several gates and only one flip-flop 
to register the carry. The serially incremented output ACA is selected by selector 
324 except during the serial load or reset count cycles so that the incremented 

10 value appears in ACxF after 8 enabled clock cycles. During each of the increment 
operations EPXC also enables LCxF and with SLD low, LCO is routed to LCIN 
which is the D input to LC7F. Thus, at the end of eight enabled clock cycles, the 
original contents of LCxF is re-circulated to its staning position so the value of 
LCxF retains its original low column address. The high column address HCxF is 

15 re-circulated in the same way and the new serialized value of ACxF at ACIN is 
compared in a bit- wise fashion with the serialized high column address at HCIN. 
The serial compare function is performed by block 327 which contains only two 
flip-flops and several logic gates. One of the flip-flops keeps track of the bit-wise 
equality of the count and the high column address while they are serially presented 

20 to the comparator and the other is used to assert the ACEH output signal during 
the next eight clock cycles for which EPXC is asserted after the count in ACxF is 
equal to the high column address from HCxF. FIRST is used to initialize the serial 
compare equal indication and LAST is used to signal the required update of 
ACEH. Note that, in this way, ACEH is asserted for exactly the number of clock 

25 cycles needed to copy the low column address into the coimter. Also, ACEH is 
output and fed as an input to the control circuit to signal the end of a row. There 
is a five pixel pause in the otherwise periodic enabling of the EPXC between rows 
so that the required row specific functions may be performed. Otherwise, during 
the entire execution period for the instruction, the coliunn count is regularly 



wo 99/14943 



PCT/US98/19075 



-37- 

incremented whether a pixel value is being read or not so that the pixel count 
periodically cycles to create the periodic assertion of ACEH which is used to 
enable the row count which is cycled in a similar fashion to cycle the integration 
frame count which in tum increments the frame read count until the required 
5 number of frames have been read. The CCLK signal is asserted only when the 
count ACxF is in its rest position between the first and last clock periods of the 8 
clock sequence so that the bits registered in the column select register are not 
shifted to an incorrect position. 

Blocks 300 and 320 together generate the column address and are 

10 also cycled as just described to establish the periodic row time base. Blocks 330 
and 340 perform a similar but not identical function to generate the row addresses 
and to create periodic frame scan timing signals. The description will be directed 
to the differences. In the intended application, the second frame column offset is 
normally small but the second frame row offset is large. Therefore, the full 8 bits 

15 of SROxF are used for the second frame row offset. Since the order reversal to 
low bit first has already been done, SROxF is a simple 8 bit shift register with an 
enable which is asserted only during the serial instruction load i.e. when SLD is 
high. The function of the adder 337 to add the second frame row offset is similar 
to the function of the related circuit incorporation of block 307 for the second 

20 frame column offset and the use of the select second frame SSF signal is similar 
as is the use of RCLK to register new row select addresses into the RSOxF 
register. However, timing of when RCLK is asserted to select new rows is quite 
different. Two separate row select coimters are initialized differently but 
incremented together. Each one is separately compared to the high row address 

25 and reset to the low row address during the next 8 cycles of CLK for which the 
enable row count ERWC signal is asserted. The reset row counter RRxF, possibly 
augmented by the second frame row offset, is used to select the row to reset at the 
beginning of the integration period. The analog to digital row coimt ARxF is used 
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to select the row to gate to the capacitive holding register for analog to digital 
conversion of the pixels in the low column to high column range of the readout 
window. The reset row coimt RRxF is initialized equal to the low row address so 
its general configuration is directly analogous to the column count. The analog to 

5 digital read row ARxF count is, however, initialized to a value which is specified 
as part of the load instruction. The value given should be greater than the low row 
count written to LRxF and less than or equal to the high row count written to 
HRxF. Each count is mdividually checked for equality to the high row count when 
it is loaded and each time that it is incremented and is reset to the low row count 

10 during the incrementing operation following the one for which equality to the high 
row count was detected. The result is that the reset row and analog to digital read 
row counters follow the same ramp patterns through an identical range of row 
count values but out of phase with each other. Data selector 345 routes bus 343 
from the RRxF counter to bus 334 when the reset row RRW signal is asserted to 

15 reset a row at the beginning of the integration period. Otherwise bus 344 from the 
ARxF counter is routed to 334 to select a row to read. The reset of successive 
rows to begm the mtegration period begms at the beginning of the instraction cycle 
when the reset row address is equal to the low row address, i.e. at the top of the 
frame. The reset of rows to begin the integration period, once begun, continues 

20 imtil all of the rows in the frame or pair of frames have been reset. The reset 
operation is suspended for integral frame periods equal to the number of 
integration delay frames inserted and continues until the frame or frame pair is 
reset for the final readout cycle. The readout of rows then begins one or more 
rows later when the analog to digital row count is set equal to the low row address 

25 so that the readout is at the top of the frame. Like the reset process, the readout 
once begun continues to the end of the frame with readout temporally suspended 
for added integration delay frame periods. The instmction execution ends just after 
readout of the last row of the last frame or frame pair is completed. 
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The IFDxF register is preset to the twos complement of the sxmi of 
the number of added integration delay periods and one. That is, for no extra delay 
periods, it is set to -1 in 2's complement form and, for 2 extra integration delay 
periods between the reset and the read for each read cycle, it is set to -3 in 2' s 

5 complement form. The value in the IFDxF register is maintained by re-circulating 
it as with the low and high column and row count registers. The counter IFCxF 
is initially loaded to 1 plus the value placed in the IFDxF register and the overflow 
condition is asserted by making the mtegration frame count overflow IFCOV high 
for the next 8 clock cycles for which the enable integration frame count EIFC 

10 signal is asserted. As with the previous blocks, EIFC is asserted to increment the 
count and the assertion of the overflow indication causes a copy of the value from 
IFDxF to be pre-mcremented and copied into IFCxF. The pre-increment function 
is used so that the overflow condition can occur on the same cycle as the counter 
is loaded. The assertion of IFCOV is used to signal the end of the integration 

15 frame delay period. It is enabled once for each cyclic frame period by asserting 
the EIFC signal for 8 CLK cycles. The serial add one and overflow detector uses 
two flip-flops and several logic gates and outputs a serial bit stream at IFCA which 
is equal to 1 plus the value of the serial bit stream input at IFIN. 

The counter 373 in block 370 is set to the negative of the sum of 1 
20 and the number of times to read the frame or frame pair. It is similar to the IFCxF 
register in block 360 and the register to reset the value is not required since 
execution of the instruction is ended at the end of the read of the final frame. The 
end of the read instmction is signaled when the RFCOV overflow condition is 
asserted and the readout of the block is completed. 

25 As an option, the SOUT signal may be output by the controller and 

conveniently mput to the serial input of the microcontroller during the instruction 
write phase. The value returned will have each serial byte value incremented by 
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the add one circuit in block 373. With this small change, it is directly determined 
by the value in the 72 bit register set at the end of the execution of the immediately 
preceding instruction and thereby constitutes a good diagnostic test point to verify 
operation of a relatively large percentage of the control logic with very little 
5 additional logic. RFCOFN has the same CLK and enable as the RFCOF and has 
RFCO as its D input. It is clocked on the negative rather than the positive edge of 
the clock. The result is that SOUT leads RFCO by one half clock pulse and is the 
proper phase to send back to the microcontroller on the MISO line. 

There are a total of 72 flip-flops in the instruction register, 25 
10 additional flip-flops in additional processing registers, 14 flip-flops in the registers 
which capture and hold the row and column addresses and 13 flip-flops in the 
serial add, compare, and overflow functions for a total of 124 flip-flops. The 
added gating and bussing logic is minimal and the control and serial interface 
blocks are implemented with a relatively modest number of added parts so that this 
15 novel approach turns out to be very efficient in terms of the logic requirement for 
the function which is performed. 

Referring to FIG. 12b, the block 500 detailed in FIG. 12c provides 
the logic which interfaces with the clock and with the SPI port of the 
microcontroller 5, for example, a Motorola 68HC708XL36, shown in FIG. 10c. 
20 The interconnections with the clock and serial port of the microcontroller 5 are 
shown in block 500 and will be described in the description of the circuit of FIG. 
12c. 

The block 501 receives SYNC, RUNS, and CLK signals from the 
block 500 and counter status information from the serially structured instruction 
25 queuing and processmg blocks 502 through 505, the block 501 responds to these 
signals by issuing a group of control and logic enable signals which control the 
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overall sequencing of the processing which takes place in the blocks 502 through 
507. The block 501 also sends the clock and the combined select and start 
conversion signals to the charge redistribution analog to digital converter (A/D) 
for example, an LTC1196. The block 500 inputs and buffers the serial data line 

5 from the LTCl 196 and sends the buffered serial form of the data signal as DOUT 
and the associated clock as NDCLK to the microcontroller 5. The blocks 502 
through 507 are detailed in FIGS. Ua and lib in a way which represents the 
novel serial architecture which serves multiple functions to queue, store, and 
process the instruction. The architectural details are clearer in this representation 

10 than in a conventional set of logic diagrams. 

The block 502 is detailed in boxes 300 and 320 in FIGS. 11a and 
lib. The flip-flops in block 502 enter and queue three bytes, 24 bits, from the 
serial instruction word during the instmction entry phase for which SLD is 
asserted. These bytes are the second frame column offset SCO, the low colunm 

15 LC of the first frame, and the high colunm HC of the first frame. During the 
instruction run or execution phase for which RUNS is asserted, this same register 
set is utilized to generate the appropriate sequence of working column addresses 
which are latched as required in the block 506. The block 506 is also depicted as 
the CSOxF flip-flop set m box 300 of FIG. 1 la. Bits 0 through 4 of the SCOxF 

20 register (i.e. SCOOF, SCOIF, SC02F, SC03F, and SC04F) are used to enter a 
signed number which designates the column offset of the columns of the second 
frame relative to the corresponding columns of the first frame. Bits 5, 6, and 7 
of this word are used for special purposes. These bits are output from block 502 
on the designated lines of bus SC0[7,0] and input as SC07 (SFM), SC06 (IRR), 

25 and SC05 (IAD) to block 501 (Note that these signals go under the SFM, IRR, 
and IAD designations in FIGS. 12a-12g). SC07 (SFM) is set in the instruction 
word to skip processing and readout of the first frame and to process only the 
second frame of the frame pair. The instmction execution and integration times are 
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shortened because of the roughly halved number of processing steps when SFM 
is set. SC06 (IRR) is set to inhibit reset of the rows at the start of the integration 
period and SC05 (IAD) is set to inhibit readout of the rows during execution. 
Setting of IAD or IRR inhibits the stated function without changing the timing of 
5 the instruction execution. 

Block 503 is detailed in boxes 330 of FIG. 11a and 340 of FIG. 
lib. The flip-flops in block 503 enter and queue four bytes, 32 bits, from the 
serial instruction word during the instruction entry phase for which SLD is 
asserted. These bytes are the second frame row offset SRO, the low row LR of the 

10 first frame, the initial row count value for the row to read with the A to D 
converter AR, and the high row HR of the first frame. During the instruction run 
or execution phase for which RUNS is asserted, this same register set is utilized 
to generate the appropriate sequence of row addresses which are latched as 
required in block 507. The block 507 is also depicted as the RSOxF flip-flop set 

15 in box 332 of FIG. 1 la. The logic generates and latches both the row numbers of 
the rows to reset at the beginning of the integration period as required and of the 
rows to read at the end of the respective integration periods. 

The block 504 is detailed in box 360 of FIG. 1 lb. The flip-flops in 
block 504 enter and queue 1 byte, 8 bits, from the serial instruction word during 
20 the instruction entry period for which SLD is asserted. During the instruction run 
or execution phase for which RUNS is asserted, this byte controls the number of 
zero or more integration frame delay periods which are inserted between reset of 
the rows of a frame and their readout to provide more "exposure" time for the 
pixel sites to accumulate light induced charge. 

25 The block 505 is detailed m box 370 of FIG. 1 lb. The flip-flops in 

block 505 enter and queue 1 byte, 8 bits, from the serial instruction word during 
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the instruction entry period for which SLD is asserted. During the instruction run 
or execution phase for which RUNS is asserted, tiiis byte controls the number of 
successive readings of the frame or frame pair which are taken and serially 
returned to the microcontroller as 8 bit digitized readings which indicate the light 
5 level received at the respective pixel sites. 

FIG. 12c is a logic diagram of the circuit which implements block 
500 of FIG. 12b. In block 500 as in the other blocks in these diagrams, the signal 
names inside the blocks which are adjacent to a pin are the names used in the 
sub-block which implements the function, the circuit of FIG. 12c in this case. The 
10 names on the wiring paths extemal to the package are the names used for the 
interconnections at that level. In most cases, the names are identical, but there are 
some signal name changes from one diagram to another when they are not. For 
example the signal name SC07 of FIG. 12b is changed to SFM in block 501 and 
its associated FIGS. 12d-12g. 

15 FIG. 12c implements the block 500 of FIG. 12c. The 

microcontroller 5 (FIG. 10c) is connected to the ASIC XC4003E by five signal 
paths. As will be apparent from the discussion, simple modifications allow this 
number to be reduced to four, with a fifth connection which may be optionally 
provided to implement an optional diagnostic check. The serial peripheral interface 

20 (SPI) port of the microcontroller 5 consists of four signal connections, three of 
which are bi-directional lines which are directly interconnected to corresponding 
pins for each device on the serial SPI bus which, in the present embodiment, 
consists of only the microcontroller 5 and the ASIC XC4003E. 

The three bi-directional bus lines are the master in slave out 
25 (MISO), the master out slave in (MOSI), and the serial peripheral serial clock 
(SPSCLK). Additionally, each device connecting to the port has an active low 
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slave select (SS) input designated SS for the microcontroller and XSS for the ASIC 
XC4003E. The SS pin to the microcontroller 5 is groimded and this requirement 
being met, programming options may be used to place the microcontroller's 5 
internal SPl port in either the master or slave mode. The active low XSS pin is 

5 driven low by the microcontroller 5 to place it in the slave mode. In a normal 
instruction to read a frame or frame pair one or more times, the microcontroller 
5 places its internal SPI port in the master mode and places the port of the ASIC 
XC4003E in the slave mode by drivmg the XSS line low. The microcontroller 5 
then serially transmits the nine byte, 72 bit, instmction to the ASIC XC4003E. In 

10 the design shown, if more bits are transmitted, the last 72 transmitted are the ones 
captured. The instmction bytes are transmitted most significant bit in each byte 
first and the order first to last is read frame count, integration frame delay, high 
row, A/D row start count, low row, second frame row offset, high column, low 
column, and the second frame column offset. During the instruction transmission 

15 phase, the master SPI port of the microcontroller 5 drives the MOSI line with the 
data and the SPSCLK line with the data clock. After the instmction is transmitted, 
the microcontroller 5 changes its intemal SPI unit to the slave mode preparing the 
program to receive the bytes of pixel data and then drives the XSS line high which 
places the ASIC XC4003E in the master mode. In response, the ASIC XC4003E 

20 enters the RUNS state and transmits the digitized pixel data back to the 
microcontroller 5 serially as the voltages from the individual pixels are digitized. 
The ASIC XC4003E then goes into an idle mode until another instmction is 
received from the microcontroller 5. The ASIC XC4003E is the master and the 
microcontroller 5 the slave during the RUNS mode so the ASIC XC4003E drives 

25 the MOSI line with the data from the A/D readings of the pixels and the SPSCLK 
Ime with the data clock. In fact, after the instmction is complete and END is 
asserted, the ASIC XC4003E contmues to drive the SPSCLK and MOSI as a 
master in the non-transmitting state until the microcontroller 5 drives XSS low. If 
XSS is driven low before an instmction is complete, it is aborted. 
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The SPI port of the microcontroller 5 has a number of clock modes, 
the one used is that for which SPSCLK signal is high in the idle states. A pull-up, 
programmed in the ASIC XC4003E for the SPSCLK pin assures that the SPSCLK 
stays high when neither SPI is in the master mode. The first enabled clock state 
should be high when either the microcontroller 5 SPI or the SPI in the ASIC is 
made the master and more than one SPI port should never be made masters at the 
same time. The clock is further configured so that the first transition is the falling 
edge at which time the data is driven onto the MOSI line. The slave clocks the 
data in on the subsequent rising edge of the clock. A careful verification that the 
transition from one clock source to another is made properly in both directions 
without dropping clocked data bits or introducing unwanted bits should be 
performed. 

During the instruction transmit mode, the ASIC XC4003E drives 
the MISO line with the data shifted out from execution of the previous instraction. 
The microcontroller 5 or a separate piece of test equipment may optionally be 
programmed to read and verify this data as the next instruction is loaded. It 
constitutes a relatively good check of the function of the ASIC logic and its 
intercoimection with the microcontroller 5 since a large part of the data path and 
the microcontroller communication on both the sending and receiving end must 
function properly for the expected 72 bit value to be retumed. If the feature is not 
used, the MISO pin does not need to be connected. 

Referring again to FIG. 12c, the OSC clock signal is driven by the 
microcontroller 5 crystal oscillator which for this application is anticipated to be 
operated in a range of 8 to 16 megahertz. However, frequencies from a wide range 
may be used. OSC enters at pad 513 and the frequency is divided by two 
producing a 50 percent duty cycle at nominally 4 to 8 megahertz at OSCD. OSCD 
is unintenupted in normal operation. The active low XSS signal driven from the 
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micrpcontroUer 5 is input at pad 514 and the signal is sampled by a D flip-flop 511 
clocked by OSCD to give NSSP, which is the synchronized present look for the 
XSS input and the value is shifted through flip-flop 512 to give NSSL which is the 
last look for the XSS input. When the XSS input has been low for at least two 

5 consecutive samples, the LOAD signal from the gate 515 is high. This is 
equivalent to the slave state for the ASIC XC4003E and MST from gate 516 is 
low. In this condition, the SPSCLK input from the microcontroller 5 at pin 521 
is gated through 517, 519, and clock buffer 520 to drive the main CLK signal for 
the ASIC XC4003E. Thus, in this mode, the system clock has one positive clock 

10 edge for each bit of the instmction word which is serially entered into the MOSI 
pad 524 of the ASIC XC4003E from the microcontroller 5 and is routed to the 
DIN pin of the serial interface block 500. As shown in FIG. 12b, the data in 
signal DIN changes name to SRCV at the output pin of block 500 and is routed 
from the SRCV output pin of block 500 to the SRCV input pin of block 502 which 

15 is the input point for the 72 bit long instmction word. In FIG. 12c, with LOAD 
high and MST low, buffers 522 and 523 are disabled so that SPSCLK and MOSI 
are not driven from the ASIC XC4003E during the LOAD operation. Buffer 525 
is driven with LOAD high so SOUT from block 505 in FIG. 12a is routed through 
the SOUT pin of block 500 to drive the MISO retum line to the microcontroller 

20 5. In block 505, the signal NSSP is driven by a negative edge triggered flip-flop 
to establish the proper phase for the data driven onto MISO. When the 
microcontroller drives XSS from low to high, the signal NSSP is driven high one 
clock cycle before NSSL so that the output of 528 goes high for one clock cycle. 
This high is gated through 529 to the D input of 527 which causes RUNS to go 

25 high. RUNS remains high due to the feedback path through 529 until 527 is reset 
by the assertion of another LOAD signal caused by driving XSS low or the normal 
end of the instmction signaled by the assertion of the ENDF signal in 530. ENDF 
is set when END is received from block 501 and cleared by the next LOAD 
instmction. When XSS is driven high, MST is asserted gating OSCD through 518, 
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519 md 520 to drive the CLK for the ASIC XC4003E. The synchronizing of 
LOAD and MST to OSCD and the one cycle clock period between the removal of 
one and the assertion of the other assures a good clock transition when the 
SPSCLK from the micro is well behaved. SYNC is asserted whenever either 
5 RUNS or LOAD is not asserted and is used for a soft initialization of block 501 
particularly to synchronize the byte timing with the bytes in the serial bit stream. 

Referring to FIGS. 12d-12e, six flip-flops 551 through 556 form 
a Johnson coimter which cycles through the 12 count values listed as counts 0 
through 11 in table 550. The counter cycles through all 12 states when RUNS is 

10 asserted and gate pairs 557 through 560 cause the counter to cycle through the first 
8 values listed as 0 through 7 in table 550 during the LOAD cycle when RUNS is 
not asserted. When neither LOAD or RUNS is asserted, SYNC is asserted clearing 
all six counter stages and holding it in the 0 state as listed in table 550. FIRST is 
asserted for the count number 0 which is always the first of the 8 clocks used to 

15 serially process information in blocks 502 through 505. LAST is asserted for the 
count number 7 which is the last of the 8 clocks used to process the information. 
CTLP is asserted for counts 8 through 1 1 and is used to inhibit the enable signals 
used for serial processing whenever these additional count states are used. CTL is 
asserted for the first of the extra count states for which the register enables are not 

20 enabled. This state is one for which all of the serial processing registers are in 
their normal rest positions and for which the "row equal high", "column equal 
high", and "overflow" states are asserted. It is the one of the 12 count cycles for 
which most of the control decisions are made as evidenced by the large number of 
flip-flops whose enables are driven by this signal. The other three counts 9 through 

25 11 which make up CTLP are provided to provide the 12 clocks per pixel required 
by the LTC1196 analog to digital converter. The coimting scheme could with 
substantial modification in the logic be made to work with only 8 counts or can 
quite easily be adjusted to any number of nine or more counts per cycle as 
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required to mesh operation of other devices such as the A to D converter with the 
operation of the rest of the circuit. During the LOAD stage, no A to D conversion 
is being performed and no specialized processing decisions are being made and 
furthermore, the CLK is derived directly from the serial input clock so there is 
5 precisely one clock per bit. This is the reason that gate pairs 557 through 560 are 
provided to cycle the counter through 8 counts per byte during the serial load 
stage. The generation of the FIRST and LAST bit indicators continue to be 
properly generated and are used in the serial instmction load operation. 

During the RUNS phase, a pause of five pixels duration is added 
10 at the beginning of the processing of each row. (The pause is a little shorter before 
the first row is processed.) This is done to provide time to perform the row reset 
at the start of the integration period for each row which is read and to generate the 
sequence of signals to gate values representative of the pixel readings for each row 
to holding capacitors at the beginning of the readout sequence for the row. A row 
15 of flip-flops 570 to 575 is configured generally as a shift register and a 1 is shifted 
through a field of zeros, the shift taking place on the clock for which the CTL 
signal enable referred to above is asserted. 

STRT is asserted at the beginning of a RUNS instruction by being 
set when RUNS is low. It causes flip-flop 591 to be set for one pixel period at the 

20 start or the RUNS sequence and this one pixel wide assertion ripples through the 
flip-flops 571 through 575. The assertion of RE4 causes flip-flop 585 to be set 
asserting PXE and a feedback of PXE causes this signal to persist imtil the end of 
row is signaled by the assertion of ACEH from the column processing block 502. 
The assertion of ACEH indicates that the last colimm hi the row is bemg 

25 processed. Flip-flop 586 inserts a one pixel width delay before assertion of PX. 
Gates 565 assert EPXC for the required 8 clock periods out of every 12 clock 
period pixel time; that is, for the 8 of the 12 clock periods for which CTLP is not 
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asserted. EPXC is used by block 502 to increment to the next column position. 
When RUNS is not asserted EPXC is continually asserted to enable the block 502 
during the instruction entry period when LOAD is asserted. 

ERWC is asserted by gate 566 to increment to the next row at the 
begiiming of the between row period when REO is asserted and only after a row 
of the second of the two frames has been processed as indicated by the requirement 
for SSFD to be asserted. Again ERWC is asserted only during the 8 of the 12 
clock periods when CTLP is not asserted. ERWC is also asserted during the 
LOAD when RUNS is not asserted. Note that when one frame only is processed, 
it is the second frame so ERWC is asserted at the end of every row. The gating of 
ERWC when RUNS is asserted is determined by gate 569 and is broken out as 
signal GINC which is additionally restricted to enable the integration frame count 
EIFC only when at the end of the last row and at the end of the row reset frame 
for which RREH is asserted. The read frame count is incremented only at the end 
of a read frame when AREH is asserted and additionally the integration frame 
count is complete as indicated by IFCOV being asserted. EIFC and EAFC are 
both asserted during the load period when RUNS is low. 

SLD is asserted when neither RUNS nor SYNC are asserted 
indicating that an instruction is being loaded. Pre-end PEND is asserted when the 
required number of frames have been read as evidenced by RFCOV being asserted 
along with AREH being asserted indicating that the last row of the last read frame 
set has been reached. The additional flip-flop asserts END after PEND is asserted 
and after the time has been provided to complete the final A to D conversion and 
send the serial data to the microcontroller 5. The timing here is fairly critical 
because too many or too few pixel clocks will cause a framing error on the serial 
transmission of the pixel readings to the microcontroller. 
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The flip-flop 580 registers the select second frame SSF indication. 
This indication is used by blocks 502 and 503 of FIG. 12a and also in the logic of 
FIGS. 12d-12e under discussion here. When SSF is asserted, the second colunm 
and second row offsets are added to the base row and column selections to process 

5 data in the second frame. In the normal mode when SFM is not asserted, SSF is 
held at 0 and the row of the first frame is processed, then without changing the 
base row value, SSF is asserted and the corresponding row of the second frame is 
processed. In this way, processing of corresponding rows of the two frames is 
interleaved and the time skew between the processing of the corresponding rows 

10 is short, being equal to the time to process one row of one of the two frames. One 
of the intended applications of the dual frame is to project two images which are 
substantially the same except for the color filtering with each having substantially 
the same position on its one of the two frames. Here the effects of motion and of 
time varying intensities are minimized by minimizing the time lapse between 

15 reading of corresponding images. Here note that an option which retains many of 
the features of the serial architecture of this invention is to separate the two 
frames, driving them in parallel as one frame with a possible registration offset 
applied to one of the two. Then the readout would also be paralleled and the 
overall row scan sequence would look more like that for scanning of a single 

20 frame. 

In FIGS. 12d-12e, when the select second frame mode SFM is 
asserted, or gate 584 causes SSF to be set continually so that the colmnn and row 
offsets are added throughout the instruction and only rows of the second frame are 
processed. In the normal dual frame mode when SFM is not asserted, exclusive 
25 or gate 582 toggles the setting of 580 and thus of SSF at the beginning of each row 
except the very first when signal 583 DREO is asserted. Flip-flop 581 outputs a 
signal SSFD which is SSF delayed by one pixel period. This delayed signal is 
useful in some of the logic operations for example in gate 569. 
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Referring to FIG. 12g, flip-flop 902 registers RSR, which is 
asserted during the frame processing intervals for which the rows are reset to begin 
the integration period. Flip-flop 910 registers ADR which is asserted during the 
period that each row is read. Gates 908 and 909 assert REN to enable the flip-flops 
controlling RSR and ADR to register a new, possibly changed, setting. REN is 
asserted at the very start of the instruction execution period when RUNS is 
asserted and STRT is still high and also at the coincident condition of the CTL 
clock period, the start of a new row as indicated by assertion of DREO, and the 
completion of the processing of the row of the second frame as indicated by the 
assertion of SSFD. 

RSR is set when STRT is asserted at the beginning of instruction 
execution and at the coincidence of the beginning of a new integration row reset 
frame (RREH is asserted for the end of the previous frame), while the readout of 
the frame is being completed (ADR is asserted), and there is another frame to read 
(RFCOV is not asserted). RSR is reset at the coincidence of conditions when the 
end of a reset frame is reached (RREH is asserted), a frame readout is not in 
progress (ADR is not asserted) and STRT is not asserted. RSR is held reset when 
the last row of the final read is in progress (PEND is asserted), or when an 
instruction is not being executed (NRUNS is asserted), or the reset of rows at the 
beginning of the integration period is inhibited (IRR is asserted). 

ADR is asserted at the coincidence of the beginning of a read frame 
(AREH is asserted) and no more integration frame periods remain (IFCOV is 
asserted). ADR is reset at the couicidence of the conditions the beginning of a read 
frame (AREH is asserted) and an integration frame delay period remains (IFCOV 
is not asserted). As with RSR, ADR is reset if PEND or NRUNS are asserted. 
ADR is not reset by the assertion of IRR but is reset by the assertion of inhibit 
A/D reading (IAD is asserted). Pads 920 through 924 are provided for 
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interconnection with the Photobit active pixel sensor which has photodiode photo 
sites. The PG pad 920 is grounded for the photodiode version the drive for which 
is illustrated but would have logic attached to generate an appropriate control 
pattern for an alternative photo gate version of the sensor. A general reset is not 
implemented in the example, the row by row reset bemg used so RESET pin 921 
is grounded. 

To reset a row, the number of the row to be reset is latched into the 
row select register block 507 of FIG. 12a by the positive edge of RCLK. The 
outputs of this register attach directly to the row select pads 18, 19, 20, 23, 24, 
25, and 26 of the ASIC XC4003E and will be routed to the corresponding row 
select pins on the image array sensor 3. The row select is latched and, after a 
settling time, the row reset RR signal is asserted on pin 924 which connects to the 
corresponding reset row input of the image array sensor 3. The reset row RR 
signal is asserted to reset rows at the begiiming of the integration period and also 
to reset a row as a part of the readout process so that differential readings of pixel 
voltages are taken by registering the pixel voltage on one of the sampling 
capacitors following the integration period, then resetting the row and registering 
the voltage on the second capacitor of the pair. One such capacitor pair is provided 
for each column of pixels and the column select function gates the voltages on the 
sampling capacitor pair for the selected column to the differential input of 
amplifier AD830 of FIG. 10a. The process just described nulls out systematic 
reading errors caused by varying zero reading voltages when the pixels are reset. 

Note that there are two different row addresses which must be 
asserted in quick succession, one of which is for the row to reset at the beginning 
of the integration period. This is done during one of the five pixel periods at the 
beginning of the row for which RRW is asserted. RRW is used by the logic of 
FIGS. 12d-12f and also is routed to block 503 of FIG. 12a to select the proper one 
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of the reset row and the A to D read row addresses. The Reset row address is 
selected to be latched into row select register 507 only when RRW is asserted, the 
row address for the A to D readout being selected otherwise. 

CCA is asserted during 6 consecutive clock periods of the 12 
5 making up the cycle of the pixel clock. CCA is asserted approximately 4 CLK 
cycles after any preceding row and column clocks and returns to zero about 2 
clock periods before any following row or column clocks making it a good signal 
to use to gate NSHS, NSHR, and RR so that the row and column selections are 
stable when these signals are asserted. 

10 The NSHS is an active low signal which gates signals from the 

selected row to holding capacitors for readout in the image array sensor 3. NSHS 
is asserted to read a row during the preprocessing period for the row when RE3 
and CCA are asserted. The reset row RR is asserted during the next pixel period 
when RE4 and CCA are asserted. Then the active low signal NSHR is asserted to 

15 gate the readings after the row is reset from the selected row to the second set of 
holding capacitors so that the differential reading can be taken as described above. 
NSHR is asserted for the reset row reading on the next pixel period when RES and 
CCA are asserted. Following this preparatory sequence at the beginning of the row 
readout, the column count is indexed from the first to the last column of the 

20 selected row gating the holding capacitor voltages for the pixel in the selected 
column to differential amplifier U6 where the difference between the normal and 
the reset pixel readings is amplified and buffered and routed to the input of a, for 
example. Linear Technology Model No. LTC1196 analog to digital converter. 

Referring to FIG. 12f, flip-flop 930 cycles RCLK high for one 
25 clock period following the one for which CTL is asserted to clock the reset row 
value into the row output register when RSR and RRW are both asserted. RCLK 



wo 99/14943 



PCT/US98/19075 



-54- 

is cycled high in a similar way to clock the row to read into the output register 
when ADR and RE2 are both asserted. Similarly, flip-flop 931 cycles CCLK high 
for one clock period following the one for which CTL is high when ADR and 
PXE are both asserted mdicating that the frame is being read and that pixels of the 
5 row are being converted. 

CCLK is asserted only when the new column address is clocked to 
the output register to output it to the Photobit sensor to select the proper pixel for 
readout. Thus, CCLK also serves to signal the start of the analog to digital 
conversion process. Flip-flop 932 serves as a pulse stretcher and CCLKP lasts for 

10 1 1 clock cycles initiated by and immediately following the CCLK pulse. Flip-flop 
933 effectively delays CCLKP and generates the active low signal NCS which is 
output on pad 938 of the ASIC XC4003E and routed to the combined chip select 
and start conversion pin of the LTC 11 96 analog to digital converter. The system 
clock CLK is routed to the LTC1196 via pad 937 of the ASIC XC4003E and 

15 converted data is returned to pad 939 of the ASIC XC4003E and clocked mto 
flip-flop 935 on the positive clock edge. This serves to reestablish the data 
transmission timing relative to CLK and NDCLK derived from gates 936 and 
supplies the 8 clock pulses transmitted with the data and ultimately used to clock 
it in to the SPI port of the microcontroller. 

20 Obviously, many modifications and variations of the present 

invention are possible in light of the above teachings. Thus, it is to be understood 
that, within the scope of the appended claims, the invention may be practiced 
otherwise than as specifically described above. 
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What Is Claimed Is: 

1 . A control system for controlling an active pixel image array 
system and controlling communication with a microcontroller, the control system 
comprising: 

5 a bidirectional interface for providing a bidirectional serial 

communication interface with said microcontroller for sending and receiving data 
from said microcontroller; 

means for controlling said active pixel array sensor; and 
means for receiving a serial instruction words from said 
10 microcontroller for controlling said active pixel image array sensor. 

2. An imaging system comprising: 

an image sensor comprised of an array of pixel sensors defining a window, each 

pixel sensor in the array at a row address and a column address, the image sensor 

having a row input for a selecting row address and a colxunn input for a selecting 
15 column address, the image sensor operative to generate a pixel output signal for 

the pixel sensor at the selecting row address and the selecting column address 

based on light incident upon the pixel sensor; 

an analog-to-digital converter in communication with the image 

sensor operative to digitize the output signal; 
20 a processor operative to generate instructions for the imaging 

system; and 

a control system in communication with the image sensor, the 
analog-to-digital converter, and the processor, the control system operative to 
(a) receive an instmction from the processor, 
25 (b) determine a first subwindow of the image sensor wmdow based on the received 
instmction, and 
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(c) determine the selecting row address and the selecting column address 
corresponding to each pixel in the subwindow. 

3. The imaging system in claim 2 wherein the processor is in 
communication with the control system through a serial bus. 

5 4. The imaging system in claim 3 wherein the serial bus is 

operative to transmit a new instruction from the processor to the control system. 

5. The imaging system in claim 4 wherein the serial bus is 
further operative to transmit a result determined by processing an immediately 
preceding instruction from the control system to the processor as the new 

10 instruction is transmitted from the processor to the control system and wherein the 
processor is operative to use the transmitted result from the control system as a 
diagnostic test point. 

6. The unaging system in claim 4 wherein the serial bus is 
further operative to transmit digitized pixel data from the control system to the 

15 processor. 

7. The imaging system in claim 3 wherein the control system 

comprises: 

a first register for holding a subwindow starting column address; 
a second register for holding a subwindow ending column address; 
20 a third register for holding a subwindow starting row address; and 

a fourth register for holding a subwindow ending row address. 

8. The imaging system in claim 7 wherein the control system 
further comprises: 
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a fifth register for holding a column offset; 
a sixth register for holding a row offset; 

wherein the control system is further operable to a determine a 
second subwindow using the position of the first subwindow, the column offset, 
5 and the row offset. 

9. The imaging system in claim 8 wherein the control system 
is further operable to switch between a single subwindow operating mode and a 
dual subwindow operating mode. 

10. The image system in claim 8 wherein the control system is 
10 further operable to alternately read the output signal of at least one pixel sensor 

from the first subwindow and read the output signal of at least one pixel sensor 
from the second subwindow. 

1 1 . The imaging system in claim 7 wherein each pixel sensor is 
operative to generate a signal by integrating light incident upon the pixel sensor 

15 and wherein each row of pixel sensors has a reset signal for initiating an 
integration period, the control system further comprising: 

a reset row counter initialized to a value determined from the 
instruction, the reset row counter periodically incremented; 

a conversion row counter initialized to a value determined from the 
20 instruction, the conversion row counter periodically incremented; 

logic for signaling a pixel sensor row reset based on the reset row 
counter value; and 

logic for signaling pixel sensor output generation based on the 
conversion row counter value; 
25 whereby the integration period is determined from the difference 

between the conversion row counter initial value and the reset row initial value. 
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12. The imaging system in claim 11 wherein the time required 
to access all rows in at least one subwindow defines a frame period, the control 
system further comprising an integration frame period counter initialized to a value 
determined from ttie instruction whereby the integration time is extended by a 

5 number of frame periods determined from the integration frame period counter. 

13. The imaging system in claim 2 wherein a set of all rows in 
at least one subwindow defines a frame, reading the frame comprises generating 
the pixel output signal for each pixel in the frame, atid wherein the control system 
further comprises a frame reads coimter initialized to a value determined from the 

10 instruction, the control system further operative to determine a number of frames 
to be read prior to ending the instraction based on the initialized value of the frame 
reads counter. 

14. The imaging system in claim 2 wherein the analog-to- 
digital converter uses successive approximation to digitize the output signal. 

15 15. A control system for controlling an image sensor wherein 

the image sensor is comprised of an array of pixel sensors, the array defining an 
image window, wherein each pixel sensor has a row address and a column 
address, and wherein each pixel sensor is operative to output a signal indicative 
of light incident on the pixel sensor over an integration period, the control system 

20 comprising: 

a serial interface for communicating with a processor; 

a plurality of registers in coromunication with the serial interface, 
the registers operative to receive an instruction from the processor; 

logic in communication with at least one register of the plurality of 
25 registers, the logic operative to determine at least one subwindow of the image 
window based on the received instruction; and 
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outputs in communication with the logic, the outputs operative to 
provide a row address and a coliman address to the image sensor corresponding to 
each pixel in the at least one sub window. 

16. The control system in claim 15 wherein the at least one 
5 subwindow includes a first subwindow, the plurality of registers comprising: 

a first register for holding a first subwindow starting column 

address; 

a second register for holding a first subwindow ending column 

address; 

10 a third register for holding a first subwmdow starting row address; 

and 

a fourth register for holding a first subwindow ending row address. 



17. The control system in claim 16 wherein the plurality of 
registers further comprises: 

15 a fifth register for holding a column offset for a second subwindow; 

a sixth register for holding a row offset for the second subwindow; 
wherein the control system is operable to determine the second 
subwindow as separated from the first subwindow by a number of columns 
indicated by the column offset and by a number of rows indicated by the row 
20 offset. 

18. The control system in claim 17 further operable to switch 
between a single subwindow operating mode and a dual subwindow operating 
mode. 
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19. The control system in claim 17 further operable to 
alternately read the output of at least one pixel sensor from the first subwindow 
and the output from at least one pixel from the second subwindow. 

20. The control system in claim 16 wherein each row of pixel 
5 sensors has a reset signal for initiatmg an integration period, the control system 

further comprising: 

a reset row counter initialized to a value determined from the 
instruction, the reset row counter periodically incremented; 

a readout row counter initialized to a value determined from the 
10 instruction, the readout row counter periodically incremented; 

logic for signaling a pixel sensor row reset based on the reset row 

counter; and 

logic for signaling output of a pixel sensor based on the readout row 

counter; 

15 whereby the integration period is determined from the difference 

between the readout row counter initial value and the reset row initial value. 

21. The control system in claim 20 wherein the time required 
to access all rows in all of the at least one subwindow defines a frame period, the 
control system further comprising an integration frame period counter initialized 

20 to a value determined from the instruction whereby the integration time is 
extended by a nmnber of frame periods determined from the integration frame 
period counter. 

22. The control system in claun 15 wherein a set of all rows in 
all of the at least one subwindow defines a frame, reading a frame comprises 

25 generating a signal from each pixel in the frame, and wherein the set of registers 
comprises a frame reads counter initialized to a value determined from the 
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instruction, the control system operative to determine the number of frames to be 
read prior to ending the instruction based on the initialized value of the frame 
reads coimter. 

23. The control system in claim 15 wherein at least two of the 
5 plurality of registers are shift registers. 

24. The control system in claim 23 wherein at least two of the 
shift registers are cascaded to form a serially contiguous register whereby 
instruction data is serially shifted into the registers. 

25. The control system in claim 24 wherein at least two of the 

10 shift registers are operable to be reconfigured for use by the logic for determining 
at least one sub window. 

26. The control system in claim 23 further comprising an 
analog-to-digital converter for digitizing the output of the pixel specified by the 
row address and the column address. 

15 27 . The control system in claim 26 wherein the analog-to-digital 

converter is a successive approximation analog-to-digital converter. 

28. The control system in claim 25 wherein the at least two of 
the reconfigurable shift registers are further operable to operate concurrently. 

29. The control system in claim 24 wherein at least one of the 
20 plurality of registers is operative to receive a portion of the instruction m parallel 

with the serially contiguous register. 
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30. The system in claim 15 wherein the serial interface is bi- 
directional. 

31. The system in claim 29 further comprising a successive 
approximation analog-to-digital converter in communication with the serial 

5 interface. 

32. The control system in claim 25 wherein the order in which 
the shift registers are cascaded eliminates separate address transmission and 
decoding. 

33. A system for controlling at least one headlamp on an 
10 automotive vehicle comprising: 

a headlamp control unit in communication with the at least one 
headlamp, the headlamp control unit operative to control the at least one headlamp 
in response to a control signal; 

a processor in communication with the headlamp control unit, the 
15 microprocessor operable to process at least one image to detect the presence of 
oncoming vehicle headlamps and leading vehicle tail lamps and to generate the 
control signal; 

an image sensor comprised of an array of pixel sensors defining a 
window, each pixel sensor in the array at a row address and a column address, the 
20 image sensor having a row input for selecting row address and a column input for 
selecting column address, the image sensor operative to generate a pixel output signal 
for the pixel sensor at the selecting row address and the selecting column address 
based on light incident upon the pixel sensor, the image sensor operable to view a 
scene in fi*ont of the automotive vehicle; 
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an analog-to-digital converter in connnunication with the image sensor 
operative to digitize the output signal and to forward the digitized output signal to 
the processor; and 

a control system in communication with the image sensor, the analog- 
to-digital converter, and the processor, the control system operative to 

(a) receive an instruction from the processor, 

(b) determine a first subwindow of the image sensor window based on the 
received instruction, and 

(c) determine the selecting row address and the selecting column address 
corresponding to each pixel in the subwindow. 

34. The imaging system in claim 33 wherein the processor is in 
communication with the control system through a serial bus. 

35. The imaging system in claim 34 wherein the serial bus is 
operative to transmit a new instruction from the processor to the control system. 

36. The imaging system in claim 35 wherein the serial bus is 
further operative to transmit a result determined by processing an immediately 
preceding instruction from the control system to the processor as the new 
instruction is transmitted from the processor to the control system and wherein the 
processor is operative to use the transmitted result from the control system as a 
diagnostic test point. 

37. The imaging system in claim 35 wherein the serial bus is 
further operative to transmit digitized pixel data from the control system to the 
processor. 
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38. The imaging system in claim 34 wherein the control system 

comprises: 

a first register for holding a subwindow starting column address; 
a second register for holding a subwindow ending colimm address; 
5 a third register for holding a subwindow starting row address; and 

a fourth register for holding a subwindow ending row address. 

39. The imaging system in claim 38 wherein the control system 
further comprises: 

a fifth register for holding a column offset; 
10 a sixth register for holding a row offset; 

wherein the control system is further operable to a determine a 
second subwindow using the position of the first subwindow, the column offset, 
and the row offset. 

40. The imaging system in claim 39 wherein the control system 
15 is further operable to switch between a single subwindow operating mode and a 

dual subwindow operating mode. 

41. The image system in claim 39 wherein the control system 
is further operable to alternately read the output signal of at least one pixel sensor 
from the first subwindow and read the output signal of at least one pixel sensor 

20 from the second subwindow. 

42. The imaging system in claim 38 wherein each pixel sensor 
is operative to generate a signal by integrating light incident upon the pixel sensor 
and wherein each row of pixel sensors has a reset signal for initiating an 
integration period, the control system further comprising: 
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a reset row counter initialized to a value determined from the 
instruction, the reset row counter periodically incremented; 

a conversion row counter initialized to a value determined from the 
instruction, the conversion row counter periodically incremented; 
5 logic for signaling a pixel sensor row reset based on the reset row 

counter value; and 

logic for signaling pixel sensor output generation based on the 
conversion row counter value; 

whereby the integration period is determmed from the difference 
10 between the conversion row counter initial value and the reset row initial value. 

43. The imaging system in claim 42 wherein the time required 
to access all rows in at least one subwindow defines a frame period, the control 
system further comprising an integration frame period counter initialized to a value 
determined from the instruction whereby the integration time is extended by a 

15 number of frame periods determined from the integration frame period counter. 

44. The imaging system in claim 33 wherein a set of all rows in 
at least one subwindow defines a frame, reading the frame comprises generating 
the pixel output signal for each pixel in the frame, and wherein the control system 
further comprises a frame reads counter initialized to a value determined from the 

20 instruction, the control system further operative to determine a number of frames 
to be read prior to ending the instmction based on the initialized value of the frame 
reads counter. 
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